Claims 

We claim: 

1. A computer-implemented method for locating regions of a target image that 
match a template image with respect to color and pattern information, the method comprising: 

performing a color matching search through the target image in order to find one or 
more color match candidate regions, wherein the one or more color match candidate regions 
match the template image with respect to color information; 

performing a luminance pattern matching search in the one or more color match 
candidate regions in order to find one or more luminance pattern match candidate regions in the 
target image; and 

calculating a hue pattern matching score for the one or more luminance pattern match 
candidate regions; 

wherein the hue pattern matching score is useable in determining regions of the target 
image that match the template image with respect to color information. 

2. The method of claim 1, further comprising: 

providing output indicating the one or more luminance pattern match candidate regions 
in the target image, wherein the one or more luminance pattern match candidate regions match 
the template image with respect to color and pattern information. 

3. The metiiod of claim 1, further comprising: 

determining one or more final match regions in the target image using the calculated hue 
pattern matching scores; and 

providing output indicating the one or more final match regions in the target image, 
wherein the one or more final match regions match the template image with respect to color and 
pattern information. 

4. The method of claim 1 , further comprising: 
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generating a color characterization of the template image prior to performing the color 
matching search; 

wherein said performing a color matching search comprises: 

generating a color characterization of a plurality of regions of the target image; 

comparing the color characterization of the template image with the color 
characterization of each of the pluraKty of regions; and 

generating a score for each of the plurality of regions indicating correlation 
between the color characterization of the template image and the color characterization of each 
of the plurality of regions. 

5. The method of claim 4, wherein said performing a color matching search further 
comprises: 

receiving user input specifying color matching search options; 

performing a first-pass color characterization search through the target image to find 
initial color match candidate areas; 

searching proximal regions of each said initial color match candidate area to find a best 
color match region for each area; 

calculating a color match score for each said best color match region; and 

generating a final Hst of color match regions based on said color match scores. 

6. The method of claim 5, wherein said searching proximal regions comprises 
using a hill-climbing technique to find said best color match region for each area. 

7. The method of claim 6, wherein said searching proximal regions to find said best 
color match region for each area using a hill-climbing technique comprises: 

using a coarse hill-climbing technique to fmd a coarse color match candidate region; and 
using a fine hill-climbing technique on the coarse color match candidate region to find 
the best color match region for each area. 

8. The method of claim 5, wherein said searching comprises searching among a 
plurality of subregions within the proximal region, wherein said searching comprises: 
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moving among the plurality of subregions using a coarse hill-climbing technique to find 
a coarse candidate subregion; and 

searching proximal subregions of the coarse candidate subregion by a fine hilklimbing 
technique to find the best color match region. 

9. The method of claim 4, further comprising: 

calculating a final color match score for the one or more luminance pattern math 
candidate regions, wherein, for each luminance pattern match candidate region, calculating the 
final color match score comprises: 

comparing the color characterization of the template image with the color 
characterization of the luminance pattem match candidate region; and 

generating the final color match score for the luminance pattem match candidate 
region indicating correlation between the color characterization of the template image and the 
color characterization of the luminance pattem match candidate region. 

10. The method of claim 9, fiirther comprising 

detemiining one or more final match regions in the target image using the hue pattem 
match score and the final color match score; and 

providing output indicating the one or more final match regions in the target image, 
wherein the one or more final match regions match the template image with respect to color and 
pattem information. 

11. The method of claim 9, wherein said performing a luminance pattem matching 
search fiirfher comprises calculating a luminance pattem match score for each of the one or 
more luminance pattem match candidate regions in the target image, the method fiirfher 
comprising: 

detemiining one or more final match regions in the target image using the luminance 
pattem match score, the hue pattem match score, and the final color match score for each of the 
one or more luminance pattem match candidate regions; and 
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providing output indicating the one or more final match regions in the target image, 
wherein the one or more final match regions match the template image with respect to color and 
pattern information. 

12. The method of claim 9, 

wherein, for each color match candidate region, the luminance pattern matching search 
comprises comparing luminance values of pixels in the template image with luminance values 
of pixels in the color match candidate region of the target image. 

13. The method of claim 9, 

wherein, for each luminance pattern match candidate region, calculating the hue pattern 
match score comprises comparing hue values of pixels in the template image with hue values of 
pixels in the luminance pattern match candidate region of the target image 

14. The method of claim 1 3 , 

wherein the pixels in the luminance pattern match candidate region exclude edge pixels 
of hue patterns in the luminance pattern match candidate region. 

15. The method of claim 9, further comprising: 

receiving user input indicating one or more colors to be masked in the color matching 

search; 

modifying color bin values for the selected masked colors in the color characterizations 
of the template image and the plurality of regions of the target image to produce respective 
masked color characterizations of the template image and the plurality of regions of the target 
image; and 

using the masked color characterizations in said color matching search. 

16. The method of claim 1 5, 

wherein said modifying color bin values comprises subtracting a value fi:om each 
masked color bin value, and wherein the resultant values are adjusted to be non-negative. 
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1 7. The method of claim 1 5, further comprising: 

zeroing color bin values for the selected masked colors in the color characterizations of 
the template image and the plurality of regions of the target image to produce respective true 
masked color characterizations of the template image and the plurality of regions of the target 
image; and 

using the true masked color characterizations in said calculating a final color match 

score. 

1 8. The method of claim 4, further comprising: 

calculating a luminance pattern match score for each of the one or more luminance 
pattern match candidate regions, wherein the luminance pattern match score indicates a 
luminance pattern match correlation between a luminance pattern match candidate region and 
the template image; 

calculating a hue pattem match score for each of the one or more luminance pattern 
match candidate regions, wherein the hue pattem match score indicates a hue pattem match 
correlation between a luminance pattem match candidate region and the template image; 

calculating a final color match score for each of the one or more luminance pattem 
match candidate regions, wherein the final color match score indicates a color match correlation 
between a luminance pattem match candidate region and the template image; 

calculating a final score for each of the one or more luminance pattem match candidate 
regions based on the luminance pattem match score, the hue pattem match score, and the final 
color match score; and 

determining one or more final match regions m the target image based upon the 
calculated final score of each of the one or more luminance pattem match candidate regions, 
wherein each of the one or more final match regions matches the template image with respect to 
color, luminance pattem, and hue pattem. 

1 9. The method of claim 1 8, 

wherein, for each luminance pattem match candidate region, calculating the final color 
match score comprises: 
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comparing the color characterization of the template image with the color 
characterization of the luminance pattern match candidate region; and 

generating the final color match score for the luminance pattern match candidate 
region indicating correlation hetween the color characterization of the template image and the 
color characterization of the luminance pattem match candidate region. 

20. The method of claim 1, further comprising: 

for each final match region found fliat matches the template image with respect to color 
and pattem information, displaying information on a graphical user interface indicating the 
location of the final match region within the target image. 

2 1 . The method of claim 1 , further comprising: 

for each final match region found that matches the template image with respect to color 
and pattem information, displaying information on a graphical user interface indicating a degree 
to which color and pattem information of the final match region matches color and pattem 
information of the template image. 

22. The method of claim 1, wherein said performing the luminance pattem matching 
search comprises: 

for each color match region found in the color matching search, performing a luminance 
pattem matching search of a proximal region proximal to the color match region in order to find 
one or more final match r^ions in the target image; 

wherein the one or more final match regions found in the luminance pattem matching 
search of each proximal region match the template image with respect to color and pattem 
information. 

23 . The method of claim 22, 

wherein said performing a luminance pattem matching search of a proximal region 
proximal to each color match region comprises performing a plurality of iterations of luminance 
pattem matching in the proximal region. 
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24. The method of claim 23, 

wherein the plurality of iterations of luminance pattern matching are performed in a 
coarse to fine manner. 

25. The method of claim 24, 

wherein said performing the plurality of iterations of luminance pattern matching in a 
coarse to fine manner uses one or more of: 

sets of sample pixels with successively smaller stability neighborhood sizes; and 
sets of sample pixels with successively smaller step sizes. 

26. The method of claim 24, wherein said searching a proximal region comprises: 
using a coarse hill-climbing technique to generate a coarse luminance pattem match 

candidate region; and 

using a fine hill-climbing technique on the coarse luminance pattem match candidate 
region to find the best luminance pattem match candidate region. 

27. The method of claim 25, wherein said searching a proximal region comprises 
searching among a plurality of subregions within the proximal region, wherein said searching 
comprises: 

moving among the plurality of subregions using a coarse hill-climbing technique to find 
a coarse candidate subregion; and 

searching proximal subregions of the coarse candidate subregion by a fine hill-climbing 
technique to find the best luminance pattem match candidate region. 

28. The method of claim 1 , 

wherein said performing the color matching search through the target image comprises: 
determining a plurality of sample regions at which to sample color information 

of the target image, wherein each sample region comprises a portion of the target image; 

for each of the plurality of sample regions, determining a measure of difference 

between color information of the sample region and the color information of the template 

image; 
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for each of the plurality of sample regions, designating tiie sample region as a 
color match region if the measure of difference between the color information of the sample 
region and the color information of flie template image is smaller than a threshold value. 

29. The method of claim 28, further comprising: 
performing a color characterization analysis of the template image; 
for each of the plurality of sample regions, performing a color characterization analysis 
of the sample region; 

wherein said determining the measure of difference between the color information of 
each sample region and the color information of the template image comprises comparing 
information obtained in the color characterization analysis of the sample region with 
information obtained in the color characterization analysis of the template image. 

- 30. The method of claim 29, 

wherein the template image and the target image each comprise a plurality of pixels; 
wherein the color characterization analysis performed for the template image and the 
color characterization analyses performed for each of the plurality of sample regions comprise: 
examining color information of at least a subset of pixels; 
assigning each examined pixel to a color category that corresponds to a portion 
of a color space; 

determining information indicative of the allocation of the examined pixels 
across color categories; 

wherein said comparing information obtained in the color characterization analysis of 
the sample region with information obtained in the color characterization analysis of the 
template image comprises comparing the allocations of the examined pixels across color 
categories for the sample region and the template image, respectively. 

3 1 . The method of claim 30, 

wherein the color characterization analysis performed for the template image comprises 
examining color information of each pixel in the template image; 
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wherein the color characterization analyses performed for each of the plurality of sample 
regions comprise examining color information of only a subset of the pixels in the sample 
region. 

32. The method of claim 30, 

wherein the color information of the template image and the color information of the 
target image comprise hue, saturation, and intensity (HSI) color information; 

wherein said examining color information of at least a subset of pixels comprises 
examining HSI information of the at least a subset of pixels; 

wherein said assigning each examined pixel to a color category that corresponds to a 
portion of a color space comprises assigning each examined pixel to a color category that 
corresponds to a portion of HSI color space. 

33. The method of claim 30 further comprising: 

wherein the color characterization analysis performed for the template image further 
comprises: 

determining one or more dominant color categories, wherein the one or more 
dominant color categories are assigned a relatively larger proportion of examined pixels, with 
respect to other color categories of the color space. 

34. The method of claim 32, 

wherein said comparing information obtained in the color characterization analysis of 
each region of the target image to information obtained in the color characterization analysis of 
the template image comprises: 

for each dominant color category, comparing the percentage of template image 
pixels assigned to the dominant color category to the percentage of target image region pixels 
assigned to that color category. 

35. The method of claim 28, wherein the target image comprises a plurality of 
pixels, the method further comprising: 
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determining a first step size to use in said determining the plurality of sample regions at 
which to sample the color information of the target image; 

wherein said determining the plurality of sample regions comprises: 

determining a plurality of regions of the target image at which to sample the 
color information of the target image, wherein the plurality of regions are each specified by 
windows into the target image, wherein windows are offset from each other by a number of 
pixels specified by the first step size. 

36. The method of claim 1, wherein the template image and the target image each 
comprise a plurality of pixels, the method further comprising: 

characterizing pattern mformation of the template image, wherein the characterizing 
comprises: 

sampling the template image to determine a first plurality of sample pixels, 
wherein the first plurality of sample pixels comprises a subset, but not all, of the pixels of the 
template image; 

wherein said performing a pattern matching search through the target image uses the 
first plurality of sample pixels. 

37. The method of claim 36, wherein the characterizing fixrfher comprises: 
performing a local stability analysis around at least a subset of the first plurality of 

sample pixels, wherein said performing the local stability analysis determines a second plurality 
of sample pixels which have a desired degree of stability, wherein the second plurality of 
sample pixels comprises a subset, but not all, of the first plurality of sample pixels; 

wherein said performing the local stability analysis operates to ensure stability of each 
of the second phiraUty of sample pixels to spatial perturbations around the sample pixel; 

wherein said performmg a pattern matching search through the target image uses the 
second plurality of sample pixels. 

38. The method of claim 37, 

wherein said performing the local stability analysis around the at least a subset of the 
first plurality of sample pixels comprises: 
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for each sample pixel, finding a neighborhood around the sample pixel where a 
value of the sample pixel correlates highly with template image pixel valies in the 
neighborhood. 

39. The method of claim 36, 

wherein each of the pixels in the template image are comprised of a plurality of 
components; and 

wherein said sampling the template image comprises sampling the template image in 
one or more component planes of the plurality of components. 

40. The method of claim 39, 

wherein each of the pixels in the template image are comprised of hue, saturation, and 
intensity components; and 

wherein said sampling the template image comprises sampling the template image in 
one or more of the hue, saturation, and intensity planes. 

4 1 . The method of claim 36, 

wherein said sampling the template image comprises sampling the template image along 
one or more rotationally invariant patiis to produce one or more sets of sample pixels; 

wherein said performing the pattern matching search uses the one or more sets of sample 
pixels to determine one or more rotated instances of the template image in the target image. 

42. A computer-implemented method for locating regions of a target image that 
match a template image with respect to color and pattern information, the method comprising: 

performing at least one of color matching and pattern matching of the target image to 
determine one or more candidate regions, wherein the one or more candidate regions match the 
template image with respect to at least one of color information and pattern information; 

calculating a hue pattern match score for at least a subset of the one or more candidate 
regions to determine one or more final match regions in the target image; and 
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providing output indicating the one or more final match regions in tiie target image, 
wherein the one or more final match regions match the template image with respect to color and 
pattem information. 

43 . The method of claim 42, 

wherein said performing at least one of color matching and pattem matching comprises 
performing at least one of color matching and luminance pattem matching. 

44. A computer-implemented method for locating regions of a target image ttot 
match a template unage with respect to color and pattem information, the method comprising: 

performing a color matching search through the target image in order to find one or 
more first candidate regions, wherein the one or more first candidate regiois match the template 
image with respect to color information; 

performing a luminance pattem matching search in at least a subset of the one or more 
first candidate regions in order to find one or more second candidate regions in the target image; 

calculating a hue pattem match score in at least a subset of the one or more second 
candidate regions to determine one or more final match regions in the target image; 

providing output indicating the one or more final match regions in the target image, 
wherein the one or more final match regions match the template image with respect to color and 
pattem information, 

45. A system for locating regions of a target image that match a template image with 
respect to color and pattem information, the system comprising: 

a processor; 

a memory medium coupled to flie processor, wherein the memory medium stores color / 

pattem matching software; 

wherein the processor is operable to execute the color / pattem matching software to: 
perform a color matching search through the target image in order to find one or more 

color match candidate regions, wherein the one or more color match candidate regions match 

the template image with respect to color information; 
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perform a luminance pattem matching search in the one or mae color match candidate 
regions in order to find one or more luminance pattem match candidate regions in the target 
image; and 

calculate a hue pattem match score for the one or more luminance pattem match 
candidate regions; 

wherein the hue pattem match score is useable in determining regions of the target 
image that match the template image with respect to color and pattem information. 

46. A memory medium comprising program instmctions for locating regions of a 
target image that match a template image with respect to color and pattem information, wherein 
the program instmctions are executable to implement: 

performing a color matching search through the target image in order to find one or 
more color match candidate regions, wherein the one or more color match candidate regions 
match the template image with respect to color infomiation; 

performing a luminance pattem matching search in the one or more color match 
candidate regions in order to find one or more luminance pattem match candidate r^ions in the 
target image; and 

calculating a hue pattem match score for the one or more luminance pattem match 
candidate regions; 

wherein the hue pattem match score is useable in determining regions of the target 
image that match the template image with respect to color information. 

47. The memory medium of claim 46, wherein said program instmction are further 
executable to implement: 

providing output indicating the one or more luminance pattem match candidate regions 
in the target image, wherein the one or more luminance pattem match candidate regions match 
the template image with respect to color and pattem information. 

48. The memory medium of claim 46, wherein said program instmction are fixrther 
executable to implement: 
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detennining one or more final match regions in the target image using the calculated hue 
pattern match scores; and 

providing output indicating the one or more final match regions in the target image, 
wherein the one or more final match regions match the template image with respect to color and 
pattern information. 

49. The memory medium of claim 46, wherein said program instruction are further 
executable to implement: 

generating a color characterization of the template image prior to performing the color 
matching search; 

wherein said performing a color matching search comprises: 

generating a color characterization of a plurality of regions of the target image; 
comparing the color characterization of the template image with the color 
characterization of each of the plurality of regions; and 

generating a score for each of the plurality of regions indicating correlation 
between the color characterization of the template image and the color characterization of each 
of the plurality of regions. 

50. The memoiy medium of claim 49, wherein said performing a color matching 
search further comprises: 

receiving user input specifying color matching search options; 

performing a first-pass color characterization search through the target image to find 
initial color match candidate areas; 

searching proximal regions of each said initial color match candidate area to find a best 
color match region for each area; 

calculating a color match score for each said best color match region; and 

generating a final list of color match regions based on said color match scores. 

51. The memory medium of claim 50, wherein said searching proximal regions 
comprises using a hill-climbing technique to find said best color match region for each area. 
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52. The memory medium of claim 51, wherein said searching proximal regions to 
find said best color match region for each area using a hilklimbing technique comprises: 

using a coarse hill-climbing technique to find a coarse color match candidate region; and 
using a fine hill-climbing technique on ttie coarse color match candidate region to find 
the best color match candidate region. 

53. The memory medium of claim 50, wherein said searching comprises searching 
among a plurality of subregions within the proximal region, wherein said searching comprises: 

moving among tiie plurality of subregions using a coarse hill-climbing technique to find 
a coarse candidate subregion; and 

searching proximal subregions of the coarse candidate subregion by a fine hill-climbing 
technique to find the best color match candidate region. 

54. The memory medium of claim 49, wherein said program instruction are further 
executable to implement: 

calculating a final color match score for the one or more luminance pattem match 
candidate regions, wherein, for each luminance pattem match candidate region, calculating the 
final color match score comprises: 

comparing the color characterization of the template image with the color 
characterization of the luminance pattem match candidate region; and 

generating the final color match score for the luminance pattem match candidate 
region indicating correlation between the color characterization of the template image and the 
color characterization of the luminance pattem match candidate region. 

55. The memory medium of claim 54, wherein said program instmction are further 
executable to implement: 

determining one or more final match regions in the target image using the hue pattem 
match score and the final color match score; and 

providing output indicating the one or more final match regions in the target image, 
wherein the one or more final match regions match the template image with respect to color and 
pattem information. 
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56. The memoiy medium of claim 54, wherein said performing a luminance pattern 
matching search further comprises calculating a luminance pattern match score for each of the 
one or more luminance pattern match candidate regions in the target image, the method forther 
comprising: 

determining one or more final match regions in the target image usuig the luminance 
pattern match score, the hue pattern match score, and the final color match score for each of the 
one or more luminance pattern match candidate regions; and 

providing output indicating the one or more final match regions in the target imagp, 
wherein the one or more final match regions match the template image with respect to color and 
pattern information. 

57. The method of claim 46, wherein said performing a luminance pattern matching 
search further comprises: 

using a coarse hill-climbing technique to search a proximal region of the color match 
candidate region to generate a coarse luminance pattem match candidate region; and 

using a fine hill-climbing technique on the coarse luminance pattern match candidate 
region to find the best luminance pattem match candidate region. 

58. The method of claim 57, wherein said searching a proximal region comprises 
searching among a plurality of subregions within the proximal region, wherein said searching 
comprises: 

moving among the plurality of subregions using a coarse hiltclimbing technique to find 
a coarse candidate subregion; and 

searching proximal subregions of the coarse candidate subregion by a fine hill-climbing 
technique to find the best luminance pattem match candidate region. 
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